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BESRL  Technical  Research  Reports  and  Technical  Research  Notes  are  intended  for 
sponsors  of  R&D  tasks  and  other  research  and  military  agencies.  Any  findings 
ready  for  implementation  at  the  time  of  publication  are  presented  in  the  latter  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recommenda¬ 
tions  for  official  action  normally  are  conveyed  to  appropriate  military  agencies 
by  briefing  or  Disposition  Form. 


The  SURVEILLANCE  SYSTEMS  research  program  of  the  U.  S.  Army  Behavioral  Science 
Research  Laboratory  has  as  its  objective  the  production  of  scientific  data  bearing  on  the 
extraction  of  information  from  surveillance  displays,  and  the  efficient  store  ie,  retrieval, 
and  transmission  of  this  information  within  an  advanced  computerized  image  interpreta¬ 
tion  facility.  Research  results  are  used  in  future  systems  design  and  in  the  development 
of  enhanced  techniques  for  all  phases  of  the  interpretation  process.  Research  is  conducted 
under  Army  RDT&E  Project  No.  2Q662704A721  "Surveillance  Systems:  Ground  Surveillance 
and  Target  Acquisition  Interpreter  Techniques,"  FY  1969  Work  Program. 

The  BESRL  Work  Unit,  "Determination  of  Interpreter  Techniques  in  a  Surveillance 
Facility,"  conducts  research  to  develop  quick-time  screening  and  interpretation  methods 
that  will  enable  an  interpretation  facility  to  process  rapidly  the  vastly  increa  ed  amounts 
and  different  kinds  of  imagery  expected  through  advanced  techniques  for  acquiring  aerial 
imagery.  The  present  publication  reports  on  experimentation  in  the  rapid  screening  of 
(real-time)  imagery  under  differing  conditions  of  intelligence  requirements,  timing,  and 
photo  scale. 
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EFFECTS  OF  INFORMATION  REQUIREMENTS.  TIMING  PROCEDURES.  AND  PHOTO  SCALE 
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BRIEF 


Requirement: 

To  determine  how  the  screening  of  real-time  imagery  for  frames  containing  priority 
targets  and  the  immediate  identification  of  targets  are  affected  by  variations  in  the 
complexity  of  the  task,  timing  procedures,  and  photo  scale. 


Procedure: 

School -trained  im  :je  interpreters  IN  *  32)  screened  imagery  to  select  frames  con¬ 
taining  priority  targets,  at  the  same  time  detecting  and  identifying  the  targets.  Half  the 
interpreters  were  to  identify  priority  targets  exclusively;  the  other  half  were  to  identify 
priority  targets  and  aluso  to  annotate  any  non-priority  military  objects  detected  while 
searching  for  priority  targets.  Another  division  of  the  interpreter  sample  was  made  for 
photo  scale,  half  working  with  imagery  at  1:3,300,  half  with  imsaery  at  1:2,200.  Half  were 
mechanically  paced  through  the  missions,  half  paced  themselves  under  a  total  time  limit. 
Experimental  conditions  were  allotted  according  to  a  23  factorial  research  design.  Per¬ 
formance  on  both  screening  and  target  identification  was  evaluated  for  completeness  ind 
accuracy. 


Findings: 

Interpreters  searching  exclusively  for  priority  targets  achieved  more  complete  selec¬ 
tion  of  priority  target  frames  and  priority  target  identification  but  were  less  accurate  in 
both  aspects  of  the  task  than  those  providing  additional  annotations. 

Interpreters  mechanically  paced  through  the  missions  were  leas  accurate,  but  no  less 
complete  in  identifying  priority  targets  than  those  under  a  total  time  limit.  In  screening, 
timing  procedures  produced  no  significant  differences. 

Scale  variation  liad  no  affect  on  the  completeness  or  accuracy  with  which  interpret¬ 
ers  selected  target  frames  or  identified  priority  targets. 

Interpreters  required  to  annotate  objects  other  than  priority  targets  (the  "priority 
plus"  requirement)  annotated  on  the  average  37  percent  of  the  non-priority  targets  in  the 
time  allotted.  Identifications  were  83  percent  correct. 


Utilization  of  Findings: 

For  the  conditions  typified  by  this  study,  photo  scales  in  excess  of  1:3,300  produce 
no  significant  improvement  of  interpreter  performance. 

It  appears  that  interpretation  facility  chiefs  can  control  the  completeness  or  accuracy 
of  performance  of  interpreters  by  varying  the  structure  of  the  task  imposed.  In  "hot  re¬ 
porting"  for  priority  targets,  the  interpreter  assigned  the  'priority  only"  requirement 
will  be  more  complete  while  the  assignment  of  the  "priority  plus'  requirement  will  result 
in  lowered  completeness  but  fewer  erroneous  responses. 
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EFFECTS  OF  INFORMATION  REQUIREMENTS,  TIMING  PROCEDURES,  AND  PHOTO  SCALE 
ON  INTERPRETER  PERFORMANCE 


There  is  frequent  need  for  immediate  information  concerning  the 
presence  or  absence,  location  or  disposition,  and  number  of  certain  high 
threat  enemy  elements  in  a  given  area.  Specific  information  about  pri¬ 
ority  targets  is  extracted  from  an  aerial  reconnaissance  mission  and 
forwarded  from  the  interpretation  facility  in  a  Hot  Report.  In  gener¬ 
ating  such  a  report,  the  image  interpreter  rapidly  scans  the  imagery, 
seeking  to  detect  and  identify  the  required  priority  targets  within  the 
time  allotted.  His  report  must  be  made  within  a  relatively  short  period 
of  time  to  insure  its  maximum  usefulness.  Obviously,  the  report  must  be 
as  complete  and  as  accurate  as  possible  in  order  to  provide  the  comnmnder 
with  the  relevant  information  upon  which  to  base  an  intelligent  and 
appropriate  plan  of  action. 

As  the  interpreter  goes  through  the  imagery  searching  for  priority 
targets,  he  often  detects  and  identifies,  to  some  level  of  exactness, 
enemy  positions  .tnd  pieces  of  materiel  that  are  non-priority  objects 
under  immediate  intelligence  requirements.  These  he  may  pass  over  with¬ 
out  making  any  record  of  his  observations.  At  some  future  time,  another 
interpreter — or  perhaps  the  same  one  who  made  the  Hot  Report  on  the 
mission--will  be  required  to  go  over  the  same  imagery  making  a  detailed 
report.  It  -ppears  desirable  to  design  a  technique  for  recording  these 
potentially  useful  identifications  so  as  to  preserve  any  information 
extracted  by  the  image  Interpreter  during  his  search  for  priority  objects. 


OBJECTIVES  OF  THE  RESEARCH 

The  general  purpose  of  this  present  study  wab  to  explore  the  effect 
of  changing  selected  aspects  of  the  work  lituation  on  the  performance  of 
the  image  interpreter.  Three  specific  objectives  were  established: 

1.  The  central  objective  was  to  determine  the  extent  to  which,  in 
the  screening  process,  peripheral  information — non-priority  information 
at  the  moment — might  be  preserved  for  future  use.  If  non-priority  ob¬ 
jects  could  be  annotated  or  recorded  in  so.se  fashion  requiring  but  little 
added  time,  the  time  required  for  the  preparation  of  more  detailed  reports 
produced  subsequent  to  the  initial  "hot  report"  might  be  reduced. 

2.  A  second  objective  was  to  determine  how  timing  procedures- -one 
aspect  of  work  met  hod --imposed  on  the  interpreter  affects  his  perform¬ 
ance.  Normally,  the  interpreter  is  under  time  pressure  in  preparing  a 
Hot  Report  and  must  pace  himself  so  as  to  complete  the  task  within  the 
allotted  time.  If  he  spends  too  much  time  on  the  early  portion  of  a 
mission,  he  may  be  unable  to  complete  the  entire  mission.  An  alterna¬ 
tive  method  by  which  the  interpreter  was  mechanically  paced  through  the 
imagery  so  as  to  complete  the  task  within  the  allotted  time  was  devised. 

It  was  hypothesized  that,  by  freeing  the  interpreter  from  the  need  to 
pace  himself  and  thereby  allowing  him  to  concentrate  wholly  on  the  detec¬ 
tion  and  identification  task,  his  performance  would  be  improved. 


?.  The  final  objective  was  to  determine  how  interpreter  perform¬ 
ance  in  the  detection  and  identification  of  priority  objects  varies  as 
a  function  of  photo  scale.  Photographic  imagery  in  which  the  same 
tetrain  appeared  at  two  different  scales  was  available  and  was  used  to 
examine  the  possibility  that  even  a  relatively  small  change  in  scale 
might  produce  a  significant  change  in  interpreter  performance  during  the 
preparation  of  a  Hot  Report.  The  study  was  not  intended  as  exhaustive, 
since  scale  has  been  included  in  other  studies. 


METHOD 


Research  Design 

The  experimental  design  was  a  P3  factorial,  the  independent  vari¬ 
ables  being  interpretation  requirement,  timing  procedure,  and  photo 
scale,  each  at  two  levels.  The  two  interpretation  requirements  differed 
in  complexity.  In  one  case,  interpreters  limited  their  search  to  pri¬ 
ority  targets.  In  the  other,  they  annotated  any  other  objects  of  poten¬ 
tial  military  interest  they  noted  while  screening  the  imagery  for  pri¬ 
ority  information.  Instructions  for  the  two  requirements  appear  below: 


"Priority  Only"  Requirement 

1.  Mark  the  location  of  all 
tanks,  AFC's,  and  SP  guns 
directly  on  the  film. 


"Priority  Plus"  Requirement 

1.  Mark  the  location  of  all  tanks, 
APC's,  and  SP  guns  directly  on 
the  film. 


Number  locations  serially, 
beginning  with  one  for  each 
target  frame. 


Number  locations  serially, 
beginning  with  one  for  each 
target  frame. 


Report  each  identification  in 
the  answer  booklet  opposite 
the  number  you  assigned  that 
object . 


Report  each  identification  in 
the  answer  booklet  opposite 
the  number  you  assigned  that 
object . 


4.  Annotate,  on  the  film,  the 
location  of  wheeled  vehicles, 
tents,  and  other  military 
objects  noted;  DO  NOT 
SPECIFICALLY  SEARCH  FOR  SUCH 
OBJECTS . 

Under  one  timing  procMure,  the  interpreter  was  given  25  minutes  to 
detect  and  identify  the  priority  targets  shown  on  a  50-frame  roll  of  film. 
He  had  to  pace  himself  so  as  to  complete  the  task  within  the  allotted 
time.  A  second  method  also  allowed  the  interpreter  25  minutes  to  com¬ 
plete  his  tiisks,  but  he  was  provided  with  a  counter  which  pulsed  every 
50  seconds,  displaying  at  intervals  a  number  indicating  the  frame  the 
interpreter  should  be  viewing  at  a  given  time.  Each  frame  was  numbered, 
and  the  interpreter  was  to  pace  himself  by  the  counter. 
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Finally,  photo  scale  was  controlled  so  that  the  imagery  used  pre¬ 
sented  the  same  ground  area  at  two  different  scales--l 0300  and  1:2200. 

Performance  was  measured  in  terms  of  the  accuracy  and  completeness 
with  which  photo  frames  containing  one  or  more  of  the  priority  targets 
were  sorted  out  from  the  rolls  of  film  and  the  priority  targets  detected 
and  identified.  Where  interpreters  were  to  annotate  the  imagery  with  re¬ 
spect  to  non-priority  military  objects,  in  addition  to  priority  targets, 
completeness  and  accuracy  for  non-priority  military  objects  were  assessed. 


E>q>erimental  Materials 

Imagery.  The  performance  measures  used  to  evaluate  the  effects  of 
the  treatment  variables  were  based  on  four  50-frame  rolls  of  positive 
transparencies.  The  imagery  covers  various  portions  of  terrain  around 
and  including  Camp  Drum,  New  York.  Each  roll  differs  from  the  others  in 
coverage  and  target  distribution.^  Multiple  copies  of  each  of  the  four 
basic  rolls  were  available.  Of  the  200  frames  in  the  four  rolls,  57  con¬ 
tained  priority  targets — tanks,  Army  personnel  carriers,  self-propelled 
guns,  or  some  combination  of  these.  Table  1  gives  details  of  the  imagery 
used . 

Equipment .  Each  of  16  work  stations  was  equipped  with  a  light  table 
on  which  the  film  rolls  could  be  mounted  so  that  the  interpreter  could 
search  the  frames  of  each  roll  successively.  Eight  of  the  stations  were 
furnished  with  digital  readout  displays  that  could  be  activated  by  a 
central  timer  set  to  emit  a  pulse  every  50  seconds.  Each  interpreter  was 
given  the  roll  of  film  he  was  to  screen,  pencils,  and  an  answer  booklet 
in  which  to  record  the  Identifications  he  made.  Each  man  was  also  given 
a  numbered  grid  to  be  used  in  designecing  the  location  of  the  targets  or 
objects  identified. 


Ssmple 

Sixteen  officers  and  sixteen  enlisted  trainees  comprised  the  experi¬ 
mental  sample.  These  men  and  officers  took  part  in  the  experiment  just 
prior  to  their  graduation  from  the  Image  Interpretation  Course  of  the 
United  States  Army  Intelligence  School  at  Fort  Holabird,  Maryland.  The 
enlisted  trainees  were  the  first  group  to  participate  in  the  experiment. 
They  were  randomly  assigned  to  the  eight  treatment  conditions,  two  men  to 
a  cell.  One  week  later  the  officer  trainees  were  given  the  same  tasks  to 
perform.  The  sixteen  officers  also  were  randomly  assigned  to  treatment 
conditions  with  two  officers  in  each  cell.  Each  of  the  two  experimental 
sessions  required  one  full  school  day. 

^  The  rolls  of  imagery  were  selected  from  those  in  the  Image  Interpreta¬ 
tion  Film  Library  maintained  by  the  Technical  Services  Branch  of  the 
Support  Systems  Research  Branch,  BESRL. 


Table  1 


i 


IMAGERY  CHARACTERISTICS  AND  TARGET  CONTENT  OF  PERFORMANCE  MEASURES 


Imagery  Characteristic 

Roll  Designation^ 

or  Target  Content 

T-45 

T-46 

T-47 

Biil 

T-49 

T-50 

1  T-51 

T-52 

Som 

Photo  Scale  and  Frame 
Format 

1:3,300,  6x6"  format 

■ 

■ 

■ 

■ 

■ 

i 

1:2, £00,  9xQ"  format 

B 

.... 

.... 

■ 

Code  Designation  of 

■ 

H 

■ 

m 

Area  Covered  in  Roll 

3 

B 

C 

C 

D 

l"‘.. 

Frames 

No.  with  only* 

Priority  Targets 

A 

4 

5 

14 

No.  with  only 

Non- Priority  Targets 

11 

10 

11 

ii 

43 

No.  with  both 
Priority  and  Non- 
Priority  Targets 

5 

7 

5 

< 

5 

23 

No.  of  Non-Target 

50 

29 

31 

30 

120 

Total  Frames  per  Roll 

50 

50 

50 

50 

Targets 

Priority: 

Tanks 

29 

23 

14 

25 

91 

APC's 

0 

r- 

£ 

t 

2 

t 

c 

- 

6 

SP  Guns 

1 

4 

7 

0 

12 

Total  Priority 

30 

29 

23 

27 

109 

Non- Priority : 

Wheeled  Vehicles 

54 

47 

46 

58 

205 

Tents 

16 

12 

14 

19 

61 

Other 

A 

1 

1 

8 

14 

Total  Non- Priority 

74 

60 

61 

85 

280 

Total  No.  of  Targets 

104 

89 

84 

112 

389 

aP»ir»  depicting  the  mm  ground  iru  at  different  acalaa  ara  T-46  and  T-4d.  T-47  aid  T-46.  and  T-4#  and  T-SO,  T-61  and 

m 

bSelo«v  dil  a  point,  a  aingta  ualua  la  givan  to  daacnba  dia  targat  con  tan  t  of  tha  two  rolla  oovaring  the  aama  araa. 
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Conduct  of  the  Experiment 

Practice  Set  ton.  Interpreters  were  given  detailed  preliminary  in¬ 
structions  that  included  a  description  of  the  purpose  of  the  experiment, 
the  nature  of  the  task,  and  method  of  recording  responses.  Eight  of  the 
1G  in  each  group  were  instructed  in  the  purpose  and  use  of  the  pacing 
equipment.  Each  interpreter  screened  a  practice  roll  of  25  frames.  The 
experimenter  resolved  any  questions  raised  and  assured  himself  that  each 
man  understood  what  he  was  to  do. 

The  Experiment  Proper.  Each  interpreter  screened  four  50-framfe  rolls 
of  photographic  transparencies.  Rolls  of  film  were  counterbalanced  so 
that,  over  the  52  interpreters,  each  roll  was  viewed  first,  second,  third, 
and  fourth  an  equal  number  of  times  to  control  for  possible  biasing  order 
effects.  The  research  design  is  shown  in  Figure  1. 


Work  Method 

Interpretation  Task 

Photo  Scale 

Self  Paced 

Machine  Paced 

Priority  Only 

Small  Scale 

2  Officers 

2  Enlisted 

Men 

2  Officers 

2  Enlisted 

Men 

Large  Scale 

2  Officers 

2  Enlisted 

Men 

2  Officers 

2  Enlisted 

Men 

Priority  Plus 

Small  Seals 

2  Officers 

2  Enlisted 

Men 

2  Officers 

2  Enlisted 

Men 

Large  Scale 

2  Officers 

2  Enlisted 

Men 

2  Officers 

2  Enlisted 

Men 

Figure  1 .  Experimental  Design  of  the  Study 
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F*.  ■*  priority-targets-only  condition,  tie  interpreter  searched 

for,  id*.  ^_xed,  and  annotated  directly  on  the  film  the  priority  targets 
specified  (tanks,  APC's,  and  SP  guns),  at  the  same  time  classifying  each 
frame  as  a  target  or  non-target  frame.  The  recording  was  done  as  follows 
for  each  frame:  As  he  identified  each  target,  he  marked  each  item  and 
numbered  each  marking  to  show  whether  the  target  identified  was  the  first, 

second,  or  n^  target  he  had  identified  on  that  frame.  Numbering  began 
anew  with  each  frame.  After  annotating  the  location  of  each  target  on 

H 
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the  film,  the  interpreter  recorded  in  his  answer  booklet  the  identifica¬ 
tion  of  the  object  and  the  number  he  had  assigned  to  it.  He  contin'.jd  in 
this  manner  until  he  had  detected  and  identified  all  priority  targets  on 
the  frame  and  then  went  on  to  the  next  until  he  had  analyzed  all  50  frames 
or  until  23  minutes  had  elapsed.  The  interpreter  working  under  the 
priority-plus  condition  followed  the  same  procedure.  However,  if  he 
detected  a  non-priority  object,  he  annotated  the  film  by  1)  circling  any 
object  seen  as  a  wheeled  vehicle,  2)  drawing  a  triangle  around  any  object 
seen  as  a  tent,  3)  making  a  check  mark  beside  any  non- identifiable  mili¬ 
tary  object.  These  identifications  were  recorded  in  the  answer  booklet 
along  with  those  of  priority  targets.  Classification  of  a  frame  as  a 
target  frame  was  inferred  from  the  identification  of  a  target  within  the 
frame  as  a  priority  target. 

Upon  completion  of  the  task--or  at  the  end  of  25  minutes--each  in¬ 
terpreter  went  back  to  the  beginning  of  the  roll  and  recorded  the  grid 
location  of  each  target  or  object  he  had  annotated,  using  the  numbered 
grid  furnished.  The  grid  locations  were  used  for  scoring  purposes  only. 

For  interpreters  working  under  the  pacing  procedure,  there  was  no 
requirement  that  they  work  on  a  given  frame  until  the  number  of  the  next 
frame  appeared.  They  were,  however,  cautioned  against  falling  too  far 
behind . 


Dependent  Variables 

Screening  Performance 
Accuracy : 


Completeness : 


Target  Identification 


Accuracy 

(priority  targets): 


Completeness 
(priority  targets): 


Ratio  of  number  of  frames  correctly 
reported  to  contain  priority  targets 
to  the  total  number  of  frames  reported 
to  contain  priority  targets 
(correct  +  incorrect). 

Ratio  of  number  of  frames  correctly 
reported  to  contain  priority  targets 
to  the  total  number  of  frames  actually 
containing  priority  targets  (37). 


Ratio  of  number  of  priority  targets 
correctly  identified  to  total  number 
of  priority  identifications  made 
(correct  +  incorrect). 

Ratio  of  number  of  priority  targets 
correctly  identified  to  the  total 
number  of  priority  targets  contained 
in  the  imagery  (109). 
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Accuracy  Ratio  of  number  of  non-priority  mili- 

(non-priority  objects):  tary  objects  correctly  identified  to 

total  number  of  non-priority  objects 
identified  (correct  +  incorrect). 

Completeness  Ratio  of  number  of  non-priority  mili- 

( non-priority  objects):  tary  objects  correctly  identified  to 

total  number  of  non-priority  objects 
contained  in  imagery  ( 280  ) . 


Analysis  of  Variance 

For  both  screening  and  priority  target  identification,  completeness 
and  accuracy  scores  on  task  requirements  of  two  degrees  of  complexity, 
under  two  timing  procedures,  and  on  imagery  of  two  scales  were  subjected 
to  a  three-way  analysis  of  variance.  Treatment  effects  associated  with 
statistically  significant  results  are  indicated  in  the  sumnary  of  analy¬ 
sis  of  variance  tables  given  in  the  Appendix. 


RESULTS 

Screening  Performance 

Table  2  shows  the  mean  number  of  target  frames  correctly  classified 
for  each  variation  in  information  requirements,  timing  procedure,  and 
scale.  Overall,  the  average  interpreter  correctly  designated  approxi¬ 
mately  26  of  the  37  target  frames.  Performance  was  significantly  af¬ 
fected  by  requiring  the  interpreter  to  annotate  non-priority  objects  in 
the  imagery.  The  "priority  only"  vs  "priority  plus"  variable  was  the 
only  one  affecting  performance  in  designating  target  frames. 


Table  2 


PRIORITY- FRAME 

CLASSIFICATION:  MEAN 

NUMBER 

CORRECT  AND 

COMPLETENESS  INDEX 

Independent 

Level 

Mean  No. 

Variable 

Description 

N 

Correct 

#  Correct 

Photo  Scale 

1:3-500 

16 

25.31 

•70 

1 :2,210 

16 

23.25 

.68 

Task 

Priority  Only 
Priority  Plus 

16 

16 

26.56 

24.81* 

.72 

* 

.67 

Work  Method 

Self-Paced 

Machine- Paced 

16 

16 

25.31 

25.7^ 

.68 

.70 

TOTAL 

32 

25.53 

.69 

•Difference  e<onificent  at  the  .05  level  or  lees. 
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It  appears  that  asking  the  interpreter  to  attend  to  targets  other 
than  those  of  paramount  importance  had  a  detracting  effect.  The  man 
searching  for  priority  targets  alone  selected  almost  two  more  target 
frames  than  did  his  counterpart  who  was  searching  for  the  same  priority 
targets  but  in  addition  was  marking  non-priority  targets  in  passing. 

The  finding  that  the  "priority  plus"  interpreters  found  fewer 
priority  frames  than  those  working  the  "priority  only"  requirement  does 
not  tell  the  complete  story.  Of  the  66  frames  containing  non-priority 
military  objects,  the  "priority  plus"  interpreters  detected  an  average 
of  46,  or  70  percent.  This  result  may  compensate  for  their  poorer  per¬ 
formance  in  correctly  identifying  priority  target  frames,  depending  on 
the  intelligence  requirement. 

The  finding  that  interpreters  working  under  the  "priority  only" 
instruction  found  more  priority-target  frames  must  be  considered  in 
terms  of  the  number  of  frames  erroneously  designated  as  priority-target 
frames.  Obviously,  an  interpreter  can  correctly  indicate  all  of  the 
priority  frames  by  the  simple  expedient  of  classifying  every  one  as  a 
priority  frame.  Such  a  strategy  would  negate  any  useful  purpose  served 
by  screening  imagery.  Table  3  shows  the  mean  number  of  wrong  responses 
made  by  interpreters  in  classifying  priority  frames  and  the  corresponding 
accuracy  scores. 


Table  3 

PRIORITY- FRAME  CLASSIFICATION:  AVERAGE  NUMBER  WRONG  AND  ACCURACY  SCORE 


Independent 

Variable 

Level 

Description 

N 

Mean  No. 
Wrong 

Accuracy 

Score 

Photo  Scale 

1:3,300 

1:2,200 

16 

16 

3-50 

2.75 

.89 

.91 

Task 

Priority  Only 
Priority  Plus 

16 

16 

4.60 

1.66 

.86 

•94* 

Work  Method 

Self-Paced 
Machine- Paced 

ie 

16 

2.00 

4.26 

•93 

.87 

TOTAL 

32 

3.12 

.90 

‘Signiftcmt  at  tha  .OS  laval  or  laaa. 
“Significint  at  tha  .01  laval  or  laaa. 


The  "priority  only"  interpreter  made  almost  three  times  as  many 
erroneous  frame  selection#  as  the  "priority  plus"  Interpreter.  These 
erroneous  frame  classifications  include  the  situation  in  which  a  frame 
containing  non-priority  targets  was  classified  as  a  target  frame  m  well 
as  the  situation  in  which  a  frame  with  no  military  targets  imaged  was 


classified  as  a  target  frame.  Interpreters  searching  exclusively  for 
priority  targets  tended  to  classify  more  frames  as  target  frames.  This 
tendency  may  be  responsible  for  the  observed  result  that  "priority  only" 
interpreters  correctly  classified  a  greater  number  of  exposures  as  target 
frames  and  in  the  process  selected  a  greater  number  of  non-priority  frame 
as  target  frames.  This  finding  is  related  to  the  observation  reported  by 
Sadacca,  Castelnovo,  and  Ranes: 

Among  interpreters  furnished  intelligence  information 
about  the  objects  for  which  they  were  to  search,  a 
larger  proportion  were  consistently  above  the  median 
in  the  correct  identification  of  objects  appearing  in 
the  photographs.  They  also  reported  more  objects 
where  no  such  objects  appeared  in  the  photographs. 

Many  of  the  differences  were  statistically  significant.^ 

In  the  present  experiment,  the  information  furnished  would  be  the  -equire 
ment  to  search  exclusively  for  tracked  vehicles. 


Identification  of  Priority  Targets 

Information  Requirements.  The  number  of  priority  targets  correctly 
identified  was  significantly  affected  by  task  requirements.  Interpreters 
searching  for  and  identifying  only  priority  targets  did  significantly 
better  than  interpreters  required  also  to  note  and  identify  non-priority 
military  objects  fTable  4).  Neither  use  of  the  pacer  nor  difference  in 
photo  scale  significantly  affected  performance. 


Table  4 

PRIORITY- TARGET  IDENTIFICATION:  AVERAGE  RIGHT  AND  COMPLETENESS  SCORE 


Independent 

Level 

Mean  No. 

Completeness 

Variable 

Description 

N 

Right 

Score 

Photo  Scale 

1:3,300 

1:2,200 

16 

16 

69.8I 

68.25 

.64 

•  63 

Task 

Priority  Only 

16 

72.81 

•67* 

Priority  Plus 

16 

6*5. 2*} 

.60 

Work  Method 

Self-Paced 

16 

72. 06 

.66 

Machine- Paced 

16 

66.00 

.61 

TOTAL 

_ 22 _ 

69-03 _ 

_ i§3 _ 

"Industss  that  diffsrsncs  is  significant  at  tha  .06  I  aval  or  lass. 


^Sadacca,  R.,  A.  Castelnovo,  and  J.  Ranes.  Human  factors  studies  in 
image  interpretation:  The  impact  of  Intelligence  information  furnished 
interpreters.  BESRL  Technical  Research  Note  117*  June  I96I. 


Interpreters  searching  sclely  for  priority  objects  may  have  been 
more  alert  to  the  cues  and  signatures  which  identify  tracked  vehicles. 
Interpreters  working  under  the  "priority  plus"  requirement  may  have  been 
unable  to  follow  to  the  letter  their  instructions  not  to  search  for  the 
non-priority  objects.  They  may  have  looked  ior  the  other  objects  as  well. 
A  second  possible  explanation  rests  on  the  difference  between  the  total 
number  of  responses  per  target  frame  for  the  two  interpreter  groups,  For 
the  "priority  only"  group,  target  frames  had  from  one  target  per  frame 
to  as  many  as  ten  targets  per  frame.  For  the  "priority  plus"  groin, 
there  were  as  many  as  22  targets  on  a  target  frame.  A  previous  hi. SRI. 
study®-  indicated  that  the  completeness  with  which  interpreters  identity 
the  targets  imaged  on  each  target  frame  is  inversely  related  to  .he 
number  of  targets  on  that  frame. 


Wrong  Identification  of  Priority  Targets 

Interpreters  instructed  to  identify  priority  objects  and  also  to 
annotate  non-priority  objects  detected  incidentally  made  fewer  wrong 
priority  identifications  than  interpreters  searching  for  and  identifying 
only  priority  objects.  Accuracy  of  performance  under  these  two  different 
sets  of  instructions  was  not  significantly  influenced.  The  "priority 
only"  requirement  resulted  in  more  correct  identifications  of  priority 
targets  and  more  erroneous  priority  target  identifications. 

The  frequency  with  which  the  interpreters  misidentified  priority 
targets  is  sunmarized  in  Table  5,  together  with  the  accuracy  scores. 

Note  that  the  "priority  plus"  interpieters  had  an  average  accuracy  of 
90  percent  for  the  identification  of  priority  targets  while  the  "priority 
only"  interpreter  group  was  but  84  percent  accurate  (Table  c>).  The  6 
percent  difference  does  not  quite  reach  the  5  percent  level  of  confidence 
established  for  the  study. 


Table  8 

PRIORITY- TARGET  IDENTIFICATION:  AVERAGE  NUMBER  WRONG  AND  ACCURACY  SCORE 


Independent 

Level 

Mean  No. 

Accuracy 

Variable 

Description 

N 

Wrong 

Score 

Photo  Scale 

1:3,500 

1:2,200 

16 

16 

11.12 

11.00 

.87 

.87 

Task 

Priority  Only 

16 

14 .50 

.84 

Priority  Plus 

16 

7.62 

•90 

Work  Method 

Self-Paced 

Machine- Paced 

16 

16 

7. 06 

15. 06** 

*91 

.82** 

TOTAL 

32 

11.06 

.87 

'Indicates  that  ditfarenca  1a 

significant  at  the  .05  level  or 

less. 

••Indicates  that  difference  it 

significant  at  the  .01  level  or 

less. 

2s  A  study  of  Rapid  Photointerpreter  Methods.  In  preparation. 
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The  result  mentioned  above  may  appear  to  be  inconsistent  with  the 
fact  that  difference  in  number  of  wrong  responses  was  statistically 
significant,  the  "priority  only"  interpreter  group  making  a  greater  number 
of  errors.  The  nature  of  the  accuracy  measure  is  responsible  for  the 

ft 

apparent  inconsistency  f  ^  ^).  For  priority  target  Identification,  the 

number  of  right  responses--and  of  wrong  responses--was  significantly 
larger  for  the  "priority  only"  interpreters.  Since  the  accuracy  index 
expresses  these  two  factors  in  ratio  form,  any  differences  between  the 
tw  interpreter  groups  would  be  depressed. 

Timing  Procedures.  The  two  work  methods  used  in  this  experiment 
produced  a  significant  difference  in  the  number  of  identification  errors 
made  by  the  interpreters.  The  interpreter  group  that  was  mechanically 
paced  through  the  imagery  made  over  twice  as  many  errors  as  did  the  self- 
paced  group  |1;5«06  to  7. 06).  This  greater  error  rate  of  the  paced  group 
resulted  in  an  accuracy  of  82  percent  while  the  self-paced  group  had  an 
accuracy  of  Ql  percent.  It  had  been  anticipated  that  by  freeing  the 
interpreter  from  the  need  to  pace  himself,  he  would  be  able  to  concentrate 
more  completely  on  the  search  for  priority  objects  and  thus  do  a  better 
job.  A  tentative  explanation  for  this  unexpected  result  is  given  below. 

The  four  ‘50-frame  film  rolls  contained  an  average  of  nine  exposures 
showing  priority  targets.  The  number  of  targets  contained  in  these  nine 
exposures  ranged  from  one  to  ten.  Suppose  the  mechanical ly-paced  group 
of  interpreters  tended  to  work  on  each  of  the  50  frames  in  a  roll  for  the 
full  y'  seconds  allowed  by  the  timer.  The  interpreter  would  have  spent 
more  time  than  was  needed  on  the  41  frames  which  contained  no  priority 
targets.  As  he  searched  these  non-target  frames  for  the  full  JO  seconds, 
he  might  detect  some  object  (man-made  or  natural  feature)  which  resembled 
a  priority  target  and  make  an  erroneous  response. 

For  those  nine  exposures  that  actually  contained  priority  targets, 
the  machine-paced  interpreter  might  do  well  witl  those  frames  containing 
few  targets  but  might  well  be  unable  to  complete  those  exposures  with  a 
high  density  of  priority  targets.  Consequently,  he  would  move  on  to  the 
next  exposure  leaving  such  frames  with  unidentified  targets. 

The  self-paced  interpreter  could  work  through  the  roll  in  a  manner 
more  familiar  to  him,  inspecting  each  successive  frame  and,  if  in  his 
jjdgment  a  frame  contained  no  priority  targets,  going  on  quickly  to  the 
next  frame.  There  was  no  temptation  to  continue  searching  the  same 
exposure  for  a  fixed  period  of  time.  Wher  he  came  to  an  exposure  with 
a  large  number  of  priority  targets,  he  could  continue  working  on  it 
until  he  felt  that  he  had  extracted  all  of  the  priority  information 
present  on  the  exposure. 

Overall  accuracy  of  identification  of  priority  targets  was  a  rela¬ 
tively  high  87  percent.  Fewer  wrong  responses  and  more  accurate  perform¬ 
ance  were  obtained  by  having  the  interpreter  pace  himself  through  the 
mission.  It  is  possible  that  greater  experience  with  mechanical  pacing 
might  train  the  interpreter  to  use  it  as  an  aid  to  better  performance. 
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Identification  of  Non-Priority  Targets 

Table  6  shows  Che  average  number  of  non-priority  targets  of  each 
type  correctly  identified  by  the  interpreters  working  under  the  "priority 
plus"  requirement,  as  well  as  number  of  misidentlfications  and  inventive 
errors.  The  average  interpreter  working  under  the  "priority  plus"  re¬ 
quirement  located  over  100  non-priority  objects  while  he  was  doing  his 
major  task  of  identifying  tanks,  APC's,  and  SP  guns.  Since  there  was  a 
total  of  280  non-priority  objects  in  the  imagery,  mean  completeness  was 
about  37  percent.  The  accuracy  of  identification  ranged  from  92  percent 
for  wheeled  vehicles  through  73  percent  for  tents  to  a  41  percent  for 
military  objects.  The  result  of  requiring  the  interpreter  to  annotate 
non-priority  military  objects  was  to  reduce  overall  accuracy.  The  number 
of  non-priority  military  objects  invented  by  the  average  interpreter 
exceeded  the  number  he  correctly  identified. 


Table  6 


MEAN  NUMBER  OF  NON- PRIORITY  TARGETS  IDENTIFIED 
BY  "PRIORITY  PLUS"  INTERPRETERS 


Non- Priority 
Target  Category 

Number 

Correct 

Classification  of  Responses 
Misidentlfications  Inventions 

Accuracy 

Wheeled  Vehicle? 

89 

1.06 

6.44 

|92 

Tents 

7.25 

.06 

2.62 

•  73 

Military  Objects 

7.88 

— 

11.31 

.41 

TOTAL 

104.13 

1.12 

20.37 

— 

The  rather  impressive  result  showing  that  the  "priority  plus"  inter¬ 
preter  identified  about  37  percent  of  the  non-priority  information  while  < 

making  priority  target  identifications  has  to  be  evaluated  against  the 
loss  of  about  7  percent  in  completeness  of  priority  target  Identification. 

This  lower  completeness  is  further  confounded  by  the  finding  that  the 
"priority  plus"  interpreter  made  a  significantly  smaller  number  of  erro¬ 
neous  identifications  than  his  "priority  only"  counterpart. 


SUMMARY  OF  RESULTS 

Screening  Performance 

Information  requirements  imposed  on  Interpreters  significantly 
affected  their  performance  in  selecting  target  frames  for  further  search. 
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1.  "Priority  only"  Interpreters  selected  more  of  the  target  framei 
than  did  "priority  plus"  Interpreters. 

2.  However,  "priority  only"  interpreters  made  more  errors  in 
selecting  target  frames.  Overall  accuracy  favored  the  "priority  plus" 
interpreter  over  the  "priority  only"  interpreter  (94$  vs  86$). 

3.  The  "priority  plus"  interpreter  located  JO  percent  of  the 
frames  containing  non-priority  targets,  and  these  designations  were 
90  percent  accurate. 


Target  Identification 

1.  "Priority  only"  interpreters  correctly  identified  more  priority 
targets  than  did  the  "priority  plus"  interpreters. 

2.  "Priority  only"  interpreters  made  more  than  twice  as  many  erro¬ 
neous  identifications  of  targets  as  did  the  "priority  plus"  interpreters. 
Overall  accuracy,  however,  did  not  differ  significantly  for  the  two 
groups . 

3.  The  "priority  plus"  interpreter  identified  over  one-third  of 
the  non-priority  military  objects  in  the  imagery,  with  an  accuracy  of 
83  percent. 

4.  The  self-paced  interpreter  made  fewer  erroneous  identifications 
than  the  mechanically  paced  interpreter  (overall  accuracy  of  91  percent 
vs  82  percent). 


CONCLUSIONS 

Differences  in  intelligence  requirements  affect  both  f r  m«  selection 
and  identification  of  priority  targets.  A  tradeoff  situation  exists.  To 
minimize  the  number  of  errors  in  classifying  frames  as  target  or  non¬ 
target  frames,  and  in  identifying  priority  targets,  the  interpreter  should 
operate  under  the  "priority  plus"  requirement,  detecting  and  annotating 
non-priority  military  objects  noted.  Such  a  requirement  will  tend  to 
reduce  the  completeness  of  information  produced  regarding  priority  target 
frames  and  identification  of  priority  targets.  Instructions  to  concen¬ 
trate  exclusively  on  target  frames  and  priority  targets  seem  to  lead 
interpreters  to  be  over- inclusive.  Obviously,  total  completeness  of 
target  frame  selection  can  be  achieved  by  the  interpreter  who  classifies 
every  frame  as  a  priority  target  frame. 

Variations  in  photo  scale  within  the  limits  established  here  have  no 
effect  on  interpreter  performance  in  selecting  target  fracas  and  identi¬ 
fying  priority  targets.  Photo  scales  in  excess  of  1:3,300  evidently  can¬ 
not  be  expected  to  produce  significant  improvement  in  interpreter  per¬ 
formance. 

Since  the  pacing  procedure  adversely  affected  accuracy  of  identifi¬ 
cation,  the  procedure  is  evidently  not  a  desirable  one. 
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APPENDIX 


SUMMARIES  OF  ANALYSES  OF  VARIANCE 


Table  A-l 

SUMMARY  OF  ANALYSIS  OF  VARIANCE  --  NUMBER  CORRECT  AND 
COMPLETENESS  INDEX  ACROSS  TREATMENT  CONDITIONS 
(Priority  Frame  Classification) 


Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F.95 

F.99 

Photo  Scale 

2.5312S 

1 

2.53125 

.3408 

4.26 

7.82 

Task 

34.03125 

1 

74.03125 

4 . 5820* 

M 

II 

Work  Method 

1.5312S 

1 

1.73125 

.2062 

II 

II 

Scale  x  Task 

.28125 

1 

.28,125 

.0379 

M 

II 

Scale  x  Method 

2.53125 

1 

2.S312S 

.3408 

11 

It 

Task  x  Method 

1.53125 

1 

1.53125 

.2062 

11 

It 

Scale  x  Task  x  Method 
Within  Groups  (e) 

TOTAL 

11.28125 

178.25000 

231.Q6375 

1 

24 

31 

11.28125 

7.42708 

1.5180 

II 

It 

*Me«ni  iiQmficantly  ditferant,  P  £  .06. 


Table  A- 

SUMMARY  OF  ANALYSIS  OF  VARIANCE  --  NUMBER  WRONG  ACROSS 
TREATMENT  CONDITIONS 
(  Priority  Frame  Classification) 


Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F.95 

F.99 

Photo  Scale 

4.500 

1 

4.500 

.4509 

4.26 

7.82 

Task 

78.12^ 

1 

78.125 

7.8288** 

It 

II 

Method 

40.500 

1 

40.500 

4.0585 

II 

II 

Scale  x  Task 

1.125 

1 

1.125 

.1127 

11 

II 

Scale  x  Method 

•  500 

1 

.500 

.0501 

II 

II 

Task  x  Method 

28.125 

1 

28.125 

2.8184 

11 

II 

Scale  x  Task  x  Method 

3.125 

1 

3.125 

.3132 

11 

II 

Within  Groups  ( e) 

239 . 500 

24 

9-979 

11 

It 

TOTAL 

305-5^0 

31 

significantly  different,  P  £  .01. 
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Table.  A-5 


SUMMARY  OF  ANALYSIS  OF  VARIANCE  --  ACCURACY  INDEX 
ACROSS  TREATMENT  CONDITIONS 
(  Priority  Frame  Classification) 


Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F.95 

F*99 

Photo  Scale 

.00378450 

1 

.00378450 

.5018 

4.26 

7.82 

Task 

.04945512 

1 

.04945512 

6.5570* 

tf 

II 

Method 

.02635512 

1 

.02633512 

3.4916 

It 

tf 

Scale  x  Task 

.00040613 

1 

.00040613 

.0538 

tl 

It 

Scale  x  Method 

.00066613 

1 

.00066613 

.0883 

If 

II 

Task  x  Method 

.01862451 

1 

.01862451 

2.4693 

II 

It 

Scale  x  Task  x  Methoc 

.00328049 

1 

.00328049 

.4349 

II 

II 

Within  Groups  (e) 

.18101600 

24 

.00754233 

TOTAL 

.28356800 

31 

*Maana  aignificantly  different,  f  < 

;  .os. 

Table  A- 4 

SUMMARY  OF  ANALYSIS  OF  VARIANCE  —  NUMBER  CORRECT 
COMPLETENESS  INDEX  ACROSS  TREATMENT  CONDITIONS 
(Priority  Target  Identification) 

AND 

Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F.95 

F*99 

Photo  Scale 

19.5313 

1 

19.5313 

.1994 

4.26 

7.82 

Task 

457.5313 

1 

457.5313 

4.6712* 

tt 

It 

Method 

294.0313 

1 

294.0313 

3.0019 

tl 

tf 

Scale  x  Task 

124 .0312 

1 

124.0312 

1.2665 

It 

It 

Scale  x  Method 

42.7812 

1 

42.7812 

.4368 

If 

It 

Task  x  Method 

22.7812 

1 

22.7812 

.2326 

II 

It 

Scale  x  Task  x  Method  19.5313 

1 

19.5313 

.1994 

tl 

It 

Within  Groups  (e) 

2350.7500 

24 

97.9479 

TOTAL 

3330.9b88 

31 

'Main*  aignifibantly  diffarant,  P  £  J06. 
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SUMMARY  OF  ANALYSIS  OF  VARIANCE  --  NUMBER  WRONG  ACROSS 
TREATMENT  CONDITIONS 
(Priority  Target  Identification) 


Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F*95 

F.  31 

Photo  Scale 

•  125 

1 

.125 

.0010 

4.26 

7.82 

Task 

378.375 

1 

378.375 

5.8305* 

II 

It 

Method 

512.000 

1 

512. COO 

7.8896** 

II 

II 

Scale  x  Task 

14.875 

1 

14.875 

.2392 

II 

It 

Scale  x  Method 

12.500 

1 

12.50 0 

.1926 

II 

tl 

Task  x  Method 

71.730 

1 

71.7  30 

1 .1056 

M 

II 

Scale  x  Task  x  Method 

•730 

1 

•  750 

.0116 

ft 

II 

Within  Groups  ( e) 

1537.500 

24 

64. 896 

TOTAL 

254  875 

31 

•Means  significantly  diffamt.  P  i  .06. 
••Maans  aignificaitly  different,  P  £  .01. 


Table  A- 6 

SUMMARY  OF  ANALYSIS  OF  VARIANCE  --  ACCURACY  INDEX  ACROSS 
’"’'EATMENT  CONDITIONS 
(Priority  Target  Identification) 


Source  of 
Variation 

Sum  of 
Squares 

df 

Mean 

Square 

F 

F.95 

F-99 

Photo  Scale 

.0000^513 

l 

.QtV  1  5}  ' 

.  TOO 

4  .26 

7.82 

Task 

.02668050 

l 

.026(.kVip; 

'  .  *  »4  > 

II 

II 

Method 

.0639031? 

l 

.063  *V1  ' 

i.M,  M2** 

II 

II 

Scale  x  Task 

.00361250 

l 

.00  361 2‘. 

.0409 

II 

It 

Scale  x  Method 

.00505012 

l 

.005050  1  2 

•7562 

II 

II 

Task  x  Method 

.00414050 

l 

.00414050 

.6200 

tl 

II 

Scale  x  Task  x  Method 

.00004050 

l 

.0000  40  5< 

.0061 

II 

It 

Within  Groups  f e) 

.16028850 

24 

.00667869 

TOT* 

.26377088 

31 

“Means  slgrifio<yi4v  Afferent.  P  <£  .01. 
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11  •  RONtORlME  MILITARY  ACTIVITY 

Office,  Chief  of  Research  and  Development, 
DA,  Wash.,  D.  C. 

PRETER  TECHNIQUES  Work  Unit  to  determine  how  the  screening  of  real-time  Imagery  la  af¬ 
fected  under  differing  conditions  of  Intelligence  requirements,  timing  procedure*,  and 
photo  scale.  Thirty-two  graduating  atudents  ©S-the  'Image  Tnterpretat ion  Course  at~the 
USA  Intelligence  School,  Fort  Holablrd,  were  subjecta  in  the  experiment.  Each  subject 
screened  four  50-fra»#  rolls  of  photographic  transparenclea  to  aelect  frames  containing] 
priority  targets,  at  the  same  time  detecting  and  Identifying  the  targeta.  The  experi¬ 
mental  design  required  half  the  interpreter*  to  search  for  and  identify  priority  target^ 
only,  the  other  half  to  identify  priority  targets  and  while  making  the  aearch  also  to 
annotat  any  non-priority  military  object!  detected.  Half  the  men  worked  with  Imagery 
at  a  1:;  500  scale,  half  with  imagery  at  1:2,200.  Half  the  interpreter*  were  machine- 
pared  through  the  tasks;  half  were  self-paced  under  a  total  time  limit  of  25  fdnute* 
for  each  task  (one  50-frame  roll).  Performance  on  both  screening  and  target  identifi¬ 
cation  was  evaluated  for  completeness  and  accuracy.  — 

Results  of  the  study  showed:  l)  Interpreters  performing  the  "priority  only"  task 
achieved  more  complete  selection  of  priority  target  frames  and  priority  target  identi¬ 
fication  but  were  less  accurate  In  both  aspects  of  the  taak  than  those  required  to  makaj 
additional  notations.  2)  Interpreters  mechsnically-psced  through  the  task  were  less  sc 
curate  but  no  less  complete  in  identifying  priority  targets  than  those  working  under 
the  self-paced  method.  Timing  procedure*  had  no  significant  effect  on  screening  per¬ 
formance.  5)  Variation*  in  photo  scale  had  no  effact  oa  completeness  or  accuracy  with 
which  the  interpreters  selected  target  frames  ox  identified  priority  targets.  Within 
the  limits  ejtabl ished  in  this  study,  photo  scales  in  excess  of  1:3, 50^  evidently  can- 
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13.  ABSTRACT  -  Continued 

not  be  expected  to  produce  significant  Improvement  of  interpreter  per¬ 
formance.  4)  Interpreters  working  under  the  "priority  plus"  requirement 
annotated  on  the  average  37  percent  of  the  non-priority  targets  in  the 
time  allotted  with  an  accuracy  of  '.3  percent. 
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